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Abstract 

A GI/M/1  queue  la  studied,  in  which  the  arrival  process  can  be 
controlled  by  accepting  or  reiect Inn  arriving  customers  or  changing  a 
toll.  An  entering  customer  receives  a (random)  reward  and  there  is  a 
holding  cost,  «fhlch  is  convex  in  the  number  of  customers  present.  We 
compare  socially  and  Individually  optimal  joining  policies  for  finite 
and  infinite  horizons,  with  and  without  discounting.  We  show  that  a 
socially  optimal  policy  is  less  likely  to  accept  a customer  than  an 
individually  optimal  policy,  and  both  policies  are  less  likely  to 
accept  as  the  nuaber  of  customers  present  increases,  the  horizon  length 
increases,  or  the  discount  rate  decreases.  The  properties  of  optimal 
congestion  tolls  are  examined  and  it  is  discovered  that  the  toll  is  not 
monotonic  when  either  the  horizon  is  finite  or  the  discount  rate  is 
positive,  whereas  it  can  be  monotonic  for  infinite-horizon,  average- 
return  problems. 


Introduction 


Th«  subject  of  this  paper  Is  6I/M/1  queue  In  tdtlch  Che  errival  process 
cen  be  controlled  by  accepting  or  rejecting  arriving  custoasrs.  There  is  s 
rendoa  reward  earned  %dien  a cuetooer  enters  the  system  and  a convex  holding 
cost  for  the  custoowrs  already  present.  We  study  the  properties  of  indivi- 
dually and  socially  optimal  joining  policies  for  finite-  end  Inflnlte-horison 
problems,  both  with  and  without  discounting.  The  properties  of  optiasl 
congestion  tolls  are  also  examined. 

Naor  [10]  was  the  first  to  compare  individually  and  socially  optimal 
policies  and  to  demonstrate  that,  left  to  their  own  devices  (that  is,  under 
an  individually  optimal  policy),  cuatomers  will  enter  the  system  more  often 
than  is  optimal  for  the  collective  of  all  customers  (that  is,  under  a sociedly 
optimal  policy).  Naor  considered  an  M/M/1  system  operating  under  a critical- 
number  policy,  with  deterministic  reward  and  linear  holding  cost  and  expected 
steady-state  net  benefit  as  the  criterion.  Naor's  results  were  extended  to 
more  general  systems  in  references  (lit  [^li  [5],  [9],  [13],  and  [14).  The 
most  relevant  references  to  the  present  paper  are  Yechiall  [13]  and  Lippman 
and  Stidham  [9].  The  former  considered  6I/M/1  systems,  but  only  in  steady- 
state  and  only  with  deterministic  rewards  and  linear  holding  costs.  The 
latter  considered  exponential  congestion  systems  (including  the  M/M/1  system 
as  a special  case),  with  random  rewards,  finite-  and  infinite-horizons,  and 
discoxmted  return  and  average  return  as  criteria.  The  present  paper  nay  be 
regarded  as  an  extension  of  both  these  references. 

In  Section  1 we  formulate  as  a semi-Markov  decision  process  the  problem 
of  finding  a socially  optimal  policy  over  a finite-horizon.  The  socially 
optimal  policy  is  shown  to  be  monotonle,  in  the  sense  that  an  arriving  customer 
is  less  likely  to  enter  as  the  number  of  customers  already  present  increases, 
the  length  of  the  horizon  increases,  or  the  interest  rats  decreases.  The 
model  differs  from  previous  ones  (e.g. , reference  [9])  in  that  it  allows  a 


non-sero  tcralnal  rward,  or  cost.  This  fssture  askss  it  posslbls,  for  ax- 
aapla,  to  aodsl  ths  situation  in  which  tha  server  continues  to  operate  until 
all  ctistooei  oresant  at  the  and  of  tha  horizon  have  bean  served. 

In  Section  2 socially  optimal  and  individually  optimal  Joining  policies 
are  conpared.  The  difference  bettfeen  the  Joining  parameters  for  ths  two 
policies  is  shown  to  equal  tha  external  affect  of  a customer's  decision  to 
enter  on  the  costs  incurred  by  future  customers.  As  expected,  this  external 
effect  is  positive,  so  that  an  individually  optimal  policy  allows  customers 
to  enter  who  should  not  enter,  if  they  took  into  account  the  costs  imposed 
thereby  on  other  customers. 

Optimal  congestion  tolls  - entrance  fees  that  Induce  individual  customers 
to  act  in  accordance  with  a socially  optimal  policy  ~ are  studied  In  Section  3. 
It  is  shown  that,  contrary  to  intuition,  the  optimal  toll  is  asymptotically 
zero  and  hence  not  monotonic  in  the  number  of  customers  in  the  system.  The 
same  result  was  discovered  by  Llppman  and  Stidham  [9]  in  the  context  of  ex- 
ponential systems,  in  which  the  system  is  observed  at  epochs  of  service 
completion  and  "null  events"  (see  Lippmae  [7])  as  well  as  arrivals.  The  reason 
for  the  phenomenon,  however,  is  somewhat  different  and  more  basic  In  the 
present  context.  Our  analysis  sheds  light  on  why  one  should  never  expect 
monotonic  tolls  in  finite-horizon  problems,  even  when  there  is  no  discounting 
and  the  server  continues  to  operate  after  the  end  of  tha  horizon. 

Infinite-horizon  problems  are  discussed  in  Section  4.  The  monotonic 
properties  of  a socially  optimal  policy  are  shown  to  carry  over  to  the 
Infinite-horizon  problem  with  discounting,  with  no  additional  restrictions 
on  the  reward  distribution  or  holding-cost  function.  Under  a slight  restric- 
tion on  the  holding-cost  function  (weaker  than  the  restrictions  In  Lippmsn  [8]) 
the  smthod  of  successive  approxismtione  is  shown  to  converge,  *ihen  there  Is  a 
terminal  cost  equal  to  the  expected  discounted  holding  coat  incurred  until  all 
customers  in  the  system  at  tha  end  of  the  horizon  are  served,  for  the 
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Inflnitc^horlzoo,  average-return  probleo  with  linear  holding-cost > *000- 
tonlclty  of  the  socially  optiiaal  policy  Is  demonstrated  using  the  mathoda 
of  Llppman  [81. 

Finally,  we  atxidy  optimal  congestion  tolls  for  Inflnlte-horlton  pr<^lems, 

both  with  and  without  discounting.  Functional  equations  and  suggestions  for  | 

: 5 

computing  optimal  tolls  are  given  for  both  eases.  In  the  discounted  case,  | 

we  show  once  again  that  the  toll  Is  asymptotically  aero  and  hence  cannot  be  i 

monotonic.  By  contrast,  in  the  average- return  case,  we  show  that  the  toll  la 
' asynptotlcally  positive  and,  at  least  in  the  M/M/1  case,  monotonically  In- 

creasing in  the  number  of  customers  present.  This  discrepancy  between  optimal 
policies  with  and  without  discounting  Implies,  among  other  things,  that  there 
is  no  strongly  (or  Blackwell)  optimal  policy, 
i 

I 

; 

I 

r 
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1.  Socially  Optl—1  Pollct— 

He  considar  a GI/M/1  quaualng  ayston  that  can  be  controlled  by 
accepting  or  rajacting  arriving  custoaara.  Specifically,  the  tlaaa 
between  the  arrivala  of  aucceaalve  cuatoaera  are  Independent  and  Idcn* 
tically  dlatrlbuted  aa  a random  variable  T,  with  0 < E{T}  < •>.  The 
aucceaalve  aervlce  tlaaa  of  cuatonera  %iho  enter  are  independent  and 
exponentially  dlatrlbuted  with  parameter  y > 0.  Aaaoclated  with  each 
cuatoaer  la  a random  reward,  or  utility  of  aervlce,  which  la  received 
If  he  entera  the  ayatem.  The  rewards  of  aucceaalve  cuatomera  are  Inde-  i 

pendent  and  Identically  dlatrlbuted  as  a random  variable  R with  E{|r|)  < ». 

There  la  also  a determlnlatle  holding  cost.  Incurred  at  rate  h(k)  while 
there  are  k customers  In  the  system,  where  h la  a convex,  non-decreasing 
function.  Costs  and  rewards  are  continuously  discounted  at  rate  a (a^O),  so 
that  the  present  value  of  x dollars  earned  at  time  t la  exp  (-at). 

The  objective  of  a social  optimizer  la  to  maximize  the  expected  a- 
dlscounted  net  benefit  (rewards  minus  holding  costs)  to  society  as  a 
whole,  over  a finite  or  Infinite  horizon.  We  formulate  the  problem  as 
a semi-Markov  decision  process  [ll],  [6],  (Sl*  In  which  the  decision  points 
are  the  times  at  which  customars  arrive  and  the  state  of  the  system  at  an 
arrival  point  Is  denoted  (l,r),  where  1 “ nuaber  of  customers  present 
(not  Including  the  customer  seeking  admittance)  and  r “ reward  of  the  customer 
seeking  admittance.  The  possible  actions  are  a ■ 1 (admit  the  customer)  and 
a ■ 0 (reject  the  ctistomer). 

Let  V (l,r)  (n  > 0)  be  the  maxlzuil  expected  a-dlscounted  net  benefit 
n,a 

to  society  as  a whole  when  a customer  with  reward  r has  just  arrived  to  a 
system  with  1 customers  already  present,  and  the  horizon  length  Is  n periods. 

That  Is  to  say,  the  system  will  operate  until  n additional  custoowra  have 
arrived  and  then  admit  no  further  customers.  We  assume  that  at  the  end  of  the 
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horlion  (chat  la,  after  the  arrival  of  Che  cusComer),  the  syatea  eanu 

a Ceniinal  benefit  depending  on  the  nunber  of  cuatonera,  1,  atiil 

In  Che  ayatem.  For  example,  the  server  might  continue  to  operate  until  all 

1 custonera  have  been  aerved,  in  which  case  U (1)  would  be  minus  the  e«- 

o ,a 

peered  a-dlscounCed  holding  cost  Incurred  until  no  customers  remain  to  be 
aerved. 

The  V (l,r)  satisfy  the  following  recursive  equations: 

O 9 QL 

(1)  V (i,r)  - max  {r  V r U ^(1)}, 

n,a  n,a  n,a 

for  n i 0,  where 

(2)  U „(1)  - -E{  e““*T»((l  - N(t))'*’)dt}  >E{e"“\  , , ((1  - R)}, 

n,a  o n**l,a 

n ^ 1,  and  {N(t),  C ^ o}  Is  a Poisson  process  with  mean  rate  u.  Independent 
of  X.  N(t)  Is  the  nunber  of  potential  service  completions  In  [0,t],  that  Is, 
Che  number  of  services  which  could  be  conqtleCed  by  time  c if  the  server  contin- 
ued to  perform  "services"  when  no  customers  were  present.  The  first  term  on 
the  right-hand  side  of  (2)  Is  minus  the  expected  a-dlscounCed  holding  cost 
Incurred  xintll  the  next  arrival  (decision)  point.  The  second  term  Is  Che 
expected  present  value  of  the  net  benefit  from  following  an  optimal  admission 
policy  from  the  next  decision  point  until  the  end  of  the  horizon. 

Let  S (1)  5 U - U (1+1) » n > 0,  1 > 0.  It  follows  Imaedlately 

DpOi  npO  tipO 

from  (1)  that  the  socially  optimal  policy  Cakes  the  form:  admlc  a customer 

when  1 ctastomers  are  already  present  and  n decision  points  remain  If  and 
only  If 

(3)  r I ^(1). 

‘ n,n 

The  remainder  of  this  section  Is  concerned  with  verifying  the  enticlpated 

monoConlclty  of  socially  optimal  policies.  Specifically,  we  show  that  a 

socially  optimal  policy  Is  less  likely  to  admit  an  arriving  cuatomer  as  1,  the 

number  of  customers  already  present,  increases  (Theorem  3),  as  n,  the  number 
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of  roulnlng  decision  points,  increases  (Theoretn  5),  or  as  a,  the  discount 
rate,  decreases  (Theorem  6). 

For  our  first  result  (Theorem  3)  we  need  the  follovlng  two  lemmas.  The 
first  Is  from  Llppman  and  Stidham  [9]  and  the  second  expresses  well-known  and 
easily  verified  properties  of  convex  and  concave  functions. 

Lemma  1.  Let  g(i)  = max{r  + f(l+l);  f(l)},  1 ■ 0,  1,  ...  . If  f(i)  Is  concave 
and  non-lncreaalng  In  1,  then  g(l)  Is  concave  and  non-lncreaslng  In  1. 

Proof.  See  Lemma  2 of  [9]. 

Lemma  2.  Let  ^(1)  “ g(f(D),  1*0,  1,  ...,  where  f(l)  is  a convex,  non- 
decreasing,  Integer-valued  function  of  1 ■■  0,  1,  ...,  with  f(0)  ^0,  and  g(j) 

Is  a concave,  non-lncreaslng  function  of  j * 0,  1 Then  ^(1)  Is  a con- 
cave function  non-lncreaslng  of  1 0,  1,  ...  . 

We  are  now  In  a position  to  prove: 

Theorem  3.  Suppose  U (1)  is  concave  and  non- increasing  In  1.  Then,  for  each 
r,  a > 0,  n > 0,  V (l,r)  Is  concave  and  non-lncreaslng  In  1.  Moreover, 

Cl 

0 4 „(i)  < „(i-n),  1 > 0. 

Proof.  The  proof  Is  by  induction  on  n.  The  case  n ■ 0 follows  Inmedlately 
from  (1),  the  hypothesis,  and  Lemma  1.  Now  suppose  n ^ 1 and  V . (l,r)  Is 

concave  and  non-lncreaslng  In  1. 

Since  h(l),  (1  - N(t))^,  and  (1  - N(T))*  are  convex  and  non-decreasing  In 

1 (for  each  fixed  value  of  t,  N(t),  T,  and  N(T)),  It  follows  from  (2)  and 

Lemma  2 that  U (1)  Is  concave  and  non-lncreaslng  In  1.  As  a consequence, 
n,a 

S (1)  la  non-negative  and  non-decreasing  In  1.  Finally,  Lemna  1 implies  that 

IlgOI 

V (l,r)  Is  concave  and  non-lncreaslng  In  1,  which  completes  the  proof. 

OgOI 

For  the  remainder  of  this  section  we  restrict  attention  to  the  special 
case  In  which  the  terminal  reward,  minus  the  expected  o-discounted 

bolding  cost  Incurred  until  the  1 customers  remaining  In  the  system  are  served. 
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That  la 


U (1)  - -E{  7 e““S((l  - N(t))'*')dt>. 
o,a  o 


Since  h(.)  is  convex  and  non-decreasing.  U (1)  is  concave  and  non- increasing 

O g Cl 

in  this  case.  The  following  corollary  is  then  an  inmediate  consequence  of 
Theorem  3. 

Corollary  4.  Suppose  given  by  (A).  Then  ^<i,r)  is  concave  and 

non-increasing  in  i and  0 4 S^  ” ^n  ^ 

Define  v (i,r)  H V (i.r)  - V (i+l,r).  It  follows  from  (1)  and 
n.o  n.a  n.a 

Theorem  3 that  (n  > 0) 


S (1) , 
n,u 


r < S (!) 
n,o 


(5)  V (i.r) 
n.a 


> r . „(i)  < r < S„  „(i+l) 

I n.a  — n.a 


i S (i+1) . 
\ n.a' 


r > 

n,a 


From  (2)  it  follows  that  (n  ^ 1) 

(6)  S„  „(i)  - -E{  e"®*"  [h((i  - NCt))"^)  - h (d  + 1 - NCt))"^)!  dt) 

n.a  o 

+E{e"“'‘^  [V  (d  - N(T))\  R)  - V (d  + 1 - N(T))"^.  R)]) 
n-l.a  n-l,a 

- -E{  e‘“*'th  (d  - N(t))'^)  - h (d  + 1 - NCt))"^)]  dt> 

+E{e"‘*\  , „(1  - N(T).  R);  N(T)  < i). 

n-igCi  ” 


Theorem  5.  Suppose  given  by  (4).  Given  a ^ 0.  1,  and  r.  the 

functions  v (i.r)  and  S (i)  are  non-decreasing  In  n > 0. 
n.a  n.a 

Proof.  The  proof  is  by  induction  on  n.  For  n “ 0.  (5)  implies  that  v^  ^(i.r) 

> S (i)  " U (i)  - U (i+1).  Combining  this  inequality  with  (6)  for 
o.a  o.a  o.a 

n ■ 1.  we  conclude  that 
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(7)  > -E{  e"®*^[h((l  - NCt))"^)  - h ((1  + 1 - N(t))'*']  dt 

+EU"“^(U^  ^(1  - N(T))-U„  „(1  + 1 - N(T))1;  N(T)  < 1) 

OfCI  O^Ol 

From  (4)  it  follows  that 

(8)  U (1)  - -E{  e"“*^  h((l  - N(t))‘‘‘)dt}  - E{  7 e’®^((i  - N(t))”*’)dt>. 

0(0  O 'P 

Let  (M'(t),  t 0)  be  a Poisson  process  with  mean  rate  p,  independent  of  T and 
{N(t) , t i 0}.  Then 

(9)  E{  7 e“®^  h((i  - NCt))"^)  dt} 

T 

- E{e‘“'^  7 e"“^‘"''^^h  ((i  - N(T)  - (N(t)  - N(T)))'*‘)dt} 

T 

- E{e““^  7 e"“^^’’^^h  ((i  - N(T)  - N*  (t-T))'^)dt} 

T 

- E{e"“’^  I e“®‘  h((i  - N(T)  - N’ (t))'*‘)dt} 

- E{E{e"®’^  I e“®*^  h((i  - N(T)  - N' (t))'*’)dt|T,N(T)}> 

- E{e'®’^  E{  I e*®*'  h((i-N(T)  - N' (t))'^)dt>} 

- -E{e’®^  * N(T))'^)). 

0,0 

Conbining  (8)  and  (9),  we  conclude  that 

(10)  U (!)  - -E{  e'®‘  h((i  - N(t))"^)  dt}  + E{e"®^  U „((!  - Nd))"*")}. 

0,0  O o»U 

Therefore, 


(11)  „(i)  - U ^(1)  - U (i  + 1) 

OyOl  OfCI  0^0 


- -E{  ^ e"®^lh((i  - N(t))‘*’)  - h ((1  + 1 - N(t))'*')ldt 
+E(e"®'^tU  „((1  - N(T))'*’)  - U ^((i  + 1 - N (T) )■*■)!} 


Together,  (7)  and  (11)  Imply  that  Sj^  ^ then  conclude 


Turning  now  to  the  general  case  of  n > 1,  suppose  v (l.r)  > v , (i,r). 

“ n,a  * n*l,a 

Then  from  (6)  it  follows  tlmt  turn  Implies  <from 

(5))  that  V - (l,r)  > v (l.r').  This  completes  the  proof  of  the  theorem. 

n+l,ci  n,a 

Theorem  6.  Suppose  given  by  (4).  Given  n,  a,  r,  the  functions 

V (l.r)  and  S (1)  are  strictly  decreasing  in  a > 0. 
n.a  n.a 

Proof . The  proof  la  by  Induction  on  n.  using  (4),  (5),  (6)  and  the  fact 
that  h(i)  is  non- decreasing  and  v^  ^(i.r)  is  non-negative. 


2.  Coap>rl«on  of  Socially  Optiaal  and 
Individually  Optimal  Joining  Pollciea 


In  this  section  ve  characterize  the  Joining  policy  of  an  Individual 

customar  who  acts  In  his  own  interest,  and  compare  It  to  a socially  optimal 

policy.  It  will  be  seen  that  an  Individually  optimal  policy  Is  lass  selective 

(admits  more  customers)  for  each  1,  n,  and  a,  than  a socially  optimal  policy. 

Let  expected  a-dlscounted  holding  cost  incurred  by  a 

customer  who  Joins  the  system  when  1 customers  are  already  present  and  n 

periods  remain  In  the  horizon.  Then  an  Individually  optimal  Joining  policy 

for  a customer  with  reward  r,  arriving  when  1 customers  are  In  the  system  and 

n periods  remain,  takes  the  form: 

enter  Iff  r > C (1). 

n,a 

Throughout  tnls  section  ve  shall  make  the  following  simplifying 
assumptions: 

(I)  the  queue  discipline  Is  flrst-ln,  flrst-out  (FIFO); 

(II)  the  holding  cost  of  each  customer  Is  proportional  to  the  length  of 

time  he  spends  In  the  system,  with  the  same  conatant  of  proportionality, 
h,  for  all  customers: 

(ill)  a customer  who  Is  present  at  the  end  of  the  horizon  remains  in  the 
system  until  he  Is  served. 

Assumption  (1)  Implies  that  a cuatomer's  holding  time  Is  not  affected  by  the 


L 


behavior  of  future  arrivals,  and  hence  his  decision  whether  or  not  to  enter 
need  not  be  based  on  any  presumptions  or  guesses  about  much  behavior.  As- 
sumption (11)  Implies  chat  the  holding  cost  per  unit  time  to  society  as  a 
whole  while  1 customers  are  present  Is  proportional  to  1;  Chat  Is,  h(l)  " h*l. 
Without  loss  of  generality,  we  assume  henceforth  that  h ■ 1. 

Assumptions  (1),  (11),  and  (111)  together  Imply  that  the  holding  cost  of 
a customer  who  enters  when  1 are  In  Che  system  Is  In  fact  Independent  of  the 
horizon  length  n.  Specifically,  let  (Sj^,  k ^ 1)  be  a sequence  of  1.1. d.  ren- 
dom  variables,  each  exponentially  distributed  with  mean  1/u,  and  set 


*1  “ ®1  • 


. . S^,  1^1. 


10 
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Theorem  7.  ia  Increasing  and  concave  In  1 ^ 0,  for  each  0, 

and  decreasing  and  convex  In  a for  each  1^0. 

Thus,  as  expected,  an  Individually  optimal  policy  admits  fewer  customers  as 

either  the  number  In  the  system  Increases  or  the  Interest  rate  decreases. 

The  comparison  between  socially  optimal  and  Individually  optimal  policies 

Is  made  easier  if  we  first  eliminate  from  V (l.r)  the  (fixed)  holding  costs 

n,a 

due  to  the  1 customers  already  in  the  system.  The  following  development 

parallels  that  in  Lippman  and  Stidham  [9]  quite  closely. 

Let  h (1)  be  the  expected  a-discounted  holding  cost  Incurred  if  there 
n 1 01 

are  1 customers  in  the  system,  n periods  remain  In  the  horizon,  and  costs  of  fu- 
ture customers  are  lenored.  By  assumption  (1),  h (1)  is  a fixed  cost,  which 

HtOi 

is  not  affected  by  the  admission  decisions  over  the  remaining  horizon.  By 
assxmiption  (111),  this  cost  includes  the  holding  cost  incurred  after  the  end 
of  the  horizon  by  any  of  the  i cuctonars  still  in  the  ayoterc  at  that  time. 

Thus,  h (1)  ■ h (1),  independent  of  n,  where  (by  assumption  (ii))  h^(l)  ■ 

QgCI  C&  C* 

E{  ^ e”®^  (1  - N(t))^  dt>.  Finally,  assumption  (i)  implies  that  C^(D  ■ 

h (i  -f  1)  - h (i),  since  the  assumption  Chat  a customer's  holding  time  Is  not 
a a 

affected  by  future  arrivals  implies  that  Che  Increment  In  fixed  coats  Incurred 
in  going  from  state  1 to  1 -f  1 la  borne  entirely  by  Che  entering  customer. 

(This  relation  also  follows  directly  from  the  expressions  for  C^(l)  and  h^(l), 


1 
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\ 

i 


\ 1 
t 

i 


t 

'ik  ' 


since  (1  + 1 - N(t))'*'  - (1  - NCc))"*"  1,  for  0 < t < and  (1  -f  I • MCt))* 

« (i  - N(t))^  - 0,  for  t ^ 

In  light  of  assunptlons  (11)  and  (111),  It  Is  appropriate  to  assuaw  that 
the  terminal  reward  for  the  social  optimizer,  U (1),  is  given  by  (4),  with 

O g Ol 

linear  holding  cost  function,  h(l)  ■ 1.  Thus  U (1)  * -h  (1).  As  a conse- 

OgQ  Cl 

quence  of  (10),  therefore,  we  have  the  Important  relation: 

(13)  h (1)  - E{  e'®*"  (1  - N(t))'*’  dt>  + E{e"“\((l  - V(T))'*")). 

a o 01 

Define  V (i,r)  - V (l,r)  + h (1) . 1 > 0,  U (1)  - U „(1)  + h„(l), 
n,a  n,a  a n,a  n,a  a 

n ^ 0.  (Hote  that  ^=0.)  V represents  the  maximum  expected  a-dlsccunted 
controllable  net  benefit,  since  h^(l)  Is  an  unavoidable,  fixed  cost.  Using  (13) 
and  the  relation  between  h^(l)  and  C^(l),  we  can  derive  the  following  recursive 
equations,  which  are  equivalent  to  (1)  and  (2): 

(14)  0 (l,r)  ■ max  {r  - C (1)  +0  (1  + 1)  • U_  (1)),  n ^0 

ngOi  d tigGi  ^»ci 

(15)  ^(1)  - E{e‘®’'  ,„((!-  NCt))"^,  R)},  n > 1. 

n,a  n~i,a 

Note  that,  in  effect,  what  Is  done  In  this  recursion  Is  to  charge  each 

customer's  entire  expected  a-discounted  holding  cost  at  the  Instant  he  enters 

the  system,  rather  than  spread  It  out  over  his  entire  stay. 

The  recursion  also  reveals  that  It  is  socially  optimal  for  a customer  to 

enter  Iff  r > C (1)  +6  (1)  - U^  „(1  + 1),  that  Is,  S (1)  - C (1)  + 6 (1) 

“ o iiga  Hga  Oga  a UgOi 

- 0 (1  + 1) . The  difference  0 (1)  - U (1  + 1) , represents  the  external 

n,o  n,a  n,a  

effect  of  the  customer's  decision  to  enter:  the  expected  a-dlscounted  cost 

that  he  Imposes  on  future  arrivals . Our  next  result  shows  that  this  external 

effect  Is  non-negative,  so  that  S (1)  > C (1) : a socially  optimal  policy 

' n,a  o 

admits  fewer  customers  than  an  Individually  optimal  policy. 

Theorem  8.  Given  o > 0,  n > 0,  1 > 0,  0,_  _(1)  >.  U_  _(1  + 1),  and  hence 

■ — w ■■  n ngW  " UgCl 

S„„(1)>C  (1). 

n,o  “ m 

12 


I 


froof.  Clearly,  0^  ^,(1)  i ^(1  + 1).  For  each  n > 1,  If 

- ^n-l,o^^  ^***"  Implies  that  0^  ^(1)  ^ 0^^  ^(1  + 1). 

Therefore,  it  suffices  to  show  that  0 (1)  > U (1  + 1)  lopllas  that 

n,a  ■*  n,a 

i,  jj(i  ■*■  1*  t)«  But  this  Is  a direct  consequence  of  (14)  and 
the  fact  that  C^(i)  is  increasing  in  i. 


I 

r 


3.  Optimal  Congastioo  Tolls 


The  socially  optimal  policy  may  be  Implemented  by  charging  each  entering 

customer  a toll,  or  entrance  fee,  and  then  allowing  him  to  act  in  his  own 

interest.  In  this  way,  the  individually  optimal  and  socially  optimal  policies 

can  be  made  to  coincide.  Our  analyeis  thus  far  indicates  that  the  optimal 

congestion  toll  is  T (1)  - S (!)  - C (1),  and  that  T (1)  >0.  The  toll 
n,a  n,o  n,a  n,a  *“ 

is  exactly  equal  to  the  external  effect,  U (1)  - U (1+1),  caused  by  the 

n,a  n,o 

customer's  decision  to  enter. 


Lippman  and  Stidham  [ 9]  examined  optimal  congestion  tolls  in  the  context 
of  exponential  congestion  systems,  in  which  the  system  is  observed  at  the 
epochs  of  departures  and  certain  null  events  as  well  as  arrivals,  and  there 
are  no  terminal  costs,  chat  is,  customers  present  at  the  end  of  Che  horixon 
depart  immediately  rather  chan  remain  in  the  system  until  their  services  are 
completed.  In  that  context  it  was  found  that,  contrary  to  intultlor , the 
optimal  congestion  toll  is  not  monotonlc  in  1.  This  same  counter-intuitive 
result  holds  in  Che  system  under  study  in  this  paper,  but,  as  we  shall  see, 
for  rather  different  reasons. 

To  examine  further  Che  propercles  of  optimal  congestion  tolls,  first 

observe  that,  since  S (1)  - C (i)  +0  (1)  - U (1+1),  (14)  is  equlv- 

n,a  ct  n,a  n,a 

alent  to 


\ „(D)  + „(i), 

QgOI  I\  1 0(  Q p CK 


where  x <*  max  (x,0).  This  relation  can  be  conlined  with  (IS)  to  yield  (n  ^ 1) 


0^  ^(1)  - E(e"“'^[R  - , rtl  - N(T))'^)]'*')  + E{e"“^  0^  . Ul  - »(!))■*■>. 

Oga  n~JL>or 

Since  T (i)  “ U (1)  - 0 (1  + 1),  we  conclude  that,  for  n > 1,  1 > 0, 

n,a  n,a  n,a 

0,0 

(16)  T^  ^(1)  - E(e"“V  , „(i.T)}  + E(e"“\  , „(1  - N(T));  N(T)  < 1), 
n,o  n— 1,0  n~i,o 
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whare  g„_i  ■ E{ - N(t))'*’) - EUR-S^_j^^((1*H  - W(t))'^)l‘*’> 


calculate  recursively  the  values  of  the  optimal  congestion  colls  for  n > 1,2, 

...  , starting  with  T (1)  “ 0.  Note  Chat  to  calculate  T (1)  one  needs  the 

0,0  n,o 

values  of  T . „(J).  for  0 4 j ^ 1+1,  and  In  general,  T . (J),  for  0 < J 

n— Xft*  n^KyO 

4 Ifk  (k  ■ 1,  2,  ....  n). 

In  the  single-server  version  of  the  exponential  congestion  syatem  studied 

by  Llppman  and  Stldhan;  [9],  It  was  discovered  chat  T (1)  • 0,  for  1+1  > n-1, 

n,o 

and  consequently  T (1)  cannot  be  monotonlc  In  1.  The  reason  for  this 
n,o 

somewhat  unsuspected  phenomenon  Is  that  In  this  case  Che  holding  times  of 
future  customers  are  not  affected  by  whether  the  customer  who  has  just  arrived 
enters  or  not.  Each  of  them.  If  he  enters,  must  remain  In  the  system  at  least 
until  the  end  of  the  horizon,  because  at  most  one  service  completion  can  occur 
In  each  period.  Moreover,  no  service  completions  are  allowed  after  Che  end 
of  the  horizon. 

In  Che  system  currently  under  study,  both  these  properties  are  absent. 

Because  Che  system  Is  observed  only  at  arrival  points  and  service  completions 
are  governed  by  a Poisson  process,  arbitrarily  many  services  can  be  completed 
In  a period.  Even  If  this  were  not  the  case,  the  decision  of  an  arriving 
customer  to  enter  or  not  would  still  affect  future  customer's  holding  times, 
since  customers  already  In  Che  system  remain  In  Che  system  until  they  are 
served,  00  matter  how  few  periods  remain.  For  both  these  reasons,  Chan,  we 

<1 

should  expect  T (1)  to  remain  positive,  no  matter  how  large  1 is. 
n,a 

On  Che  other  hand.  It  Is  clear  that,  for  any  fixed  n,  T (1)  approaches 

n,o 

zero  asymptotically  as  1 approaches  Infinity.  To  see  this,  recall  that 

T (1)  “ U (1)  - U (1+1).  Now,  It  Is  clear  from  their  definitions  that 
n,o  n,a  n,a 

both  V (l,r)  and  0 (1)  are  non-negative.  But  0 (1)  is  non- Increasing  In  1. 

n,o  n,a  n,a 


S ((1+1  - N(t»^) 
E{ 

Vl  ’ M(t))  ) 


Pr{R  > r)  dr). 


Using  (16),  (12),  and  Che  relation  S (1)  » C (1)  + T (1),  one  can 

n,a  a n,a 
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Thsrafor*,  0 (1)  approaches  a finite  limit,  and  hence  T (1)  approaches  taro, 

np  o HpO 

as  1 approaches  infinity.  We  conclude  that,  just  as  in  the  syataa  studied  by 
Llppman  and  Stidham  l9]>  it  is  impoasible  for  T (1)  to  be  monotonlcally 
Increasing  in  1.  Nor  is  ^(1)  generally  decreasing  in  1.  To  see  this, 

suppose  n ■ 1,  In  which  case 

T,  (1)  - E{e“®V  „((i  - N(T)  )'*')}. 

X p la  O p Q 

Since  T (1)  0,  S (1)  ■■  C (i),  for  all  1 > 0.  Now  assume  that 

OpCi  OpO  a ^ 

Pr{R>r}  - 1 for  r>C  (M).  Then  g <l,t)  - C ((1+1  - NCt))"*")  - C (<1-N(t))'*')  and 

a 0,0  o a 

T,  „(1)  - E{e"®^[C  ((1  + 1 - Nd))"^)  - C ((1  - N(T))'*')]} 

1,0  a a 


E{e"“’‘[C  (1  + 1 - N(T))  - C (1  - N(T))I;  H(T)  < 1} 
a u 


E(e 


-oT 


I [C  (i  + 1 - k)  - C (1  - k)]  e"‘'’^(pT)Vkl) 
, . o o 

1+1-k 


E{e 


-oT 


4 1 IfX-IC 

^ ^ — {—■ — ) -liT 

Z o + V n + y'  e 


e~^^(yT)Vk!} 


k“0 


1 i 

“ TT7  ^r+T^  E{  r ((o  + y)T)’^/kl} 


k»0 


Now  suppose  X is  exponentially  distributed  with  parameter  X.  Then 

i+1  . k 

U'' 


l.a 

Therefore 


I.  -(«  ■ rhr  <a  ■.  X . u> 


1 . O +JJ. 


(t 


t) 


E '■o  + X + y'  . 
k-0 


^1,0^^^  “ ^1,0^°^  “o  + y ^o  + X+y^  ^o  + y^  ^o  y o + X + y' 


For  any  fixed  values  of  X and  y,  the  term  in  brackets  is  positive  for  samll 


a and  negative  for  large  o.  Hence,  T-  (i)  is  not  monotonic  In  1. 

1,0 


A.  Optlnal  Policlea  for  the  Infinite  Horizon 


In  this  ssctlon  we  study  socially  optlnal  policies  for  Infinite- 
horizon  problens,  both  with  and  without  discounting.  In  contrast  to  Section  2, 
we  do  not  make  assuaptlons  (1),  (11) • and  (111),  but  rather  return  to  the 
generality  of  Section  1.  In  particular,  the  holding  ccst  rate  h(l)  Is  once 
sgaln  allowed  to  be  a general  convex,  non-decreasing  function  of  1,  the 
nuid>er  of  custooers  present. 

We  first  consider  the  Inflnlts-horlson  problem  with  discounting.  Ue 
assume  that  S “ <1.  A policy  t Is  defined  In  Che  usual  way 

(cf.  [1],[6],  [8],  [lli)  as  a sequence  of  decision  rules,  possibly  randomised 
and  possibly  dependent  on  the  history  of  the  process  as  well  as  the  current 
state,  (l,r),  for  selecting  actions  (a  0 or  1)  at  each  decision  point.  A 
stationary  policy  la  a deterministic  policy  that  always  takes  the  same  action, 

a “ f(l,r),  whenever  the  state  Is  (l,r).  Let  (l,r)  be  the  expected  a- 

dlscounted  net  benefit  over  an  Infinite  horizon  from  following  policy  ir, 
when  the  starting  state  is  (l,r).  lhat  Is,  (l,r)  is  the  e]q>ected  sum  of 
the  discounted  net  returns  (rewards  minus  holding  costs)  earned  in  all  periods 

1c  " Xg  2g  • a • • 

To  see  that  (l,r)  Is  well  defined,  first  note  that  the  positive  part  of 
the  discounted  net  return  in  period  k Is  bounded  above  by  e , where  tj^ 

and  Rj^  are  the  arrival  times  and  rewards,  respectively,  of  customer  k.  Since 
tj^  and  Rj^  do  not  depend  on  the  policy  In  effect,  this  bound  holds  for  all 
policies.  Consequently,  the  expected  discounted  positive  part  of  the  net 
return  In  period  k,  under  any  policy,  is  bounded  above  by  8 E(R  ),  where 
6 < 1 and  E{R^}  < •,  by  assumption.  It  follows  that  condition  (3)  of  Serfozo 

[1^  is  satisfied,  so  that  Serfozo' s Assumptions  (Al)  and  (A2)  hold.  Hence, 

(l,r)  Is  well  defined.  (It  may  equal  - **.) 
a 

Define  (l,r)  - s^  V*  (l,r).  (l,r)  la  the  optimal  return  function 

for  the  Infinite-horizon  problem  with  discounting.  An  application  of  Theorems 

2.1  and  2.2  of  Serfozo  (1^  then  yields: 
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Thaorea  9.  (Itt)  MtisflM  the  CuacClonel  equation 

(17)  V Cl.r)  - aa*  (r  + U (1+1),  U (1)} 

a a a 


where 

(18)  (1)  - - E{  e"®‘  h ((1  - N(t))'*’)dt}  + E{e”®^  ((l-Hd))"^) }. 

Moreover,  the  atatlonary  policy  that  accepta  a cuatoaer  Iff  r > S (1)  s 

“ a 

U^(l)  - U^(l+1)  la  optimal  aaong  all  pollclaa. 

The  next  theorem  shows  that  the  aonotonlcity  of  optimal  policies  carries 
over  from  finite  to  Infinite  horizon. 


Theorem  10.  V^(l,r)  and  U^(i)  are  concave  In  1,  so  that  S^(l)  la  non- 

decreasing in  1 and  hence  a critical-number  policy  Is  optimal. 

Proof.  Consider  a sequence  of  finite-horizon  problems,  in  which  the  teralnal 

reward,  U (1),  Is  Identically  zero.  It  follows  from  Theorem  2.2  (c)  of 

0,0 

Serfozo  [ij  that  V (l,r)  - 11m  V (l,r)  and  hence  U (1)  - 11m  U (1). 

® “ n^ 

Concavity  of  V (l,r)  and  U (1)  then  follows  from  Theorem  3. 
o a 

The  above  proof  shows  that  the  optimal  return  function,  V^(l,r),  and  the 
optimal  reward-cutoff  point,  S^(l),  can  be  approximated  by  solving  a sequence 
of  finite-horizon  problems,  with  U (1)  S 0.  The  question  remains  as  to 

O g 01 

whether  the  method  of  successive  approximations  works  for  other  terminal  reward 
functions.  (This  Is  a non-trlvlal  problem,  since  the  one-period  net  return 
Is  unbounded.)  The  following  theorem  gives  a partial  answer  to  this  question. 

Theorem  11.  Suppose  that  condition  (4)  holds:  that  Is,  U.  (1)  Is  minus  the 

■“  • ■ OgCI 

expected  a-dlscounted  holding  cost  incurred  until  the  1 customers  still  In 

the  system  are  served.  Suppose  also  that  0°  h(n)  -»■  o aa  n Then 

^(l,r)  t V (l,r),  ^(1)  t U (1),  and  S (1)  ♦ S (1),  as  n 

Hg(i  Cl  dgOi  a UgCi  a 
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1 

b. 


Proof.  For  each  (l,r)  and  n i 0,  U (1)  ^ 8 E{R*'}/(1  - 8)  < • and 

O f Cl 

i r B E{K^}/(1  - B)  < •.  Moreover,  U (1)  and  V (l,r)  arc  both 
“•'*  n,a  0(0 

nofr-decreaslng  In  n,  as  may  be  shown  by  a simple  Induction  argument  using  (1) 

and  (2).  (To  get  started,  one  must  show  that  U,  (1)  > U (1).  But 

i,a  “*  0,0 

V„  „(l,r)  >V^  (i)  implies  that  0,  ^(1)  > - E{  h((i  - N(t))'^)dt}  + ' 

OgO  OgCI  X»<*  O 

E{e  U ((1  - N(T))^)},  which  equals  U (1)  by  (10).)  Hence  both 

OgOl  OgCI 

V_  _(l,r)  and  U „(1)  monotonlcally  approach  finite  limits  as  n Moreover, 

U g d By  Q 

it  follows  from  (1)  and  (2)  that  these  limits,  say  V (i,r)  and  U*  (1),  satisfy 

Ot  01 

(17)  and  (18).  If  we  knew  that  the  solutions  to  (17)  and  (18)  were  unique, 

then  Iheorem  9 would  imply  that  (l,r)  ■ V^(i,r)  and  (1)  ■ (1),  and 

ve  would  be  done.  However,  (i,r)  and  (1)  need  not  be  the  unique  solutions 

to  (17)  and  (18),  since  the  one-period  net  return  is  not  bounded.  One  could 

establish  uniqueness  by  Imposing  further  conditions,  such  as  the  existence  of  a 

polynomial  bound  on  the  one-period  net  return,  and  then  applying  the  results  of 

Llppman  [8].  Instead  of  this  approach,  we  shall  employ  a jore  direct  argument 

to  show  that  V'  (l,r)  ■ V (l,r)  (and  hence  U'  (1)  ■ U (1)). 

a a a a 

To  this  end,  first  observe  that,  under  the  assumption  that  U (1)  is 

0,0 

given  by  (4),  V (l,r)  is  the  expected  o-dlscousted  total  net  benefit  over 
n,o 

an  infinite  horizon  from  following  a particular  policy,  namely  the  policy 

which  takes  the  optimal  actions  for  the  n-perlod  problem  at  the  first  n 

decision  points,  and  then  admits  no  further  customers.  Thus,  (l,r)  ^ 

^ (i,r).  Now  let  ^ (i,r)  be  the  optimal  expected  o-dlscounted  total 

net-benefit  over  n periods  when  the  terminal  reward  is  identically  zero.  (Ne 

know  from  the  proof  of  Theorem  10  that  V*  (i,r)  V (i,r)  and  U*  (i)  -*• 

n,a  a n,o 

(1).)  Let  ir*  be  the  policy  that  achieves  V*  ^ (i,r),  that  is,  the  policy 

that  is  optimal  for  the  n-period  problem  with  terminal  reward  identically  zero. 

Now  consider  what  happens  if  we  employ  policy  x*  for  the  n-period  problem 

when  the  terminal  reward  is  U (i),  as  given  by  (4).  The  expected  o- 

0,0 

discounted  total  net  benefit  over  n periods  from  following  «*  is  in  this  ease 


1 
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_ (i-t*)  + E_«  {«  U (X  (l»r))},  where  X (l,r)  • nuaber  of  cuetoaers 
UfCi  0|Ci  n Q 

present  at  the  end  of  the  n-perlod  horizon,  given  the  initial  state  Is  (l,r), 

and  E • {•}  denotes  the  expected  value  under  policy  n*  . But.  since  v la 

n n 

optimal  for  the  n-perlod  problem  with  terminal  reward  U (1),  we  conclude 

O p Q 

that 

(1»)  v;  ^(l.r)  + B,.  {.-»•«  ^ (X„(i.r)))  < v„  < V,  (l.r). 


How  from  the  definition  of  U (1), 

0,0 


^ i ~ OC„(l,r))} 

• IT  Cfu  n 

n 

> E{e“®“^  U (1  + n + 1)} 


0.0 


- 6®  U (i  + n + 1) 


0.0 


- - b“  E{  ^ e"“^  h ((1  n + 1 - N(t))'^)dt} 


> - 6®  h (i  + n + l)/o. 


It  follows,  therefore,  from  (10)  and  the  assumption  that  8 h(n)  0 that 

\ <^»*‘>  “ iis  -^o  ii8  '^n.o 

" V (l.r),  the  desired  result. 

Cl 

He  now  turn  our  attention  to  the  Infinite-horizon  undiscounted  problem 

(i  ■ 0)  In  which  the  objective  is  to  maximize  long-run  average  expected  return 

per  unit  time.  For  the  remainder  of  the  paper  we  assume  that  the  holding  cost 
Is  linear  : h(l)  = 1. 

Theorem  12 . ’ Suppose  the  holding  coat  Is  linear:  h(l)  = 1.  Then  the  functions 

V (l.r)  H 11m  (V  (l.r)  - U (0)),  U(l)  = llm  (U  (1)  - U (v»  are  well  defined, 

® ® a-*o4-  “ “ 


concave,  and  satisfy  the  functional  equation 


V(l,r)  - max  (r  + U(l-M),  0(1)> 


(20) 


U(l) 


- - E{  (1  - H(t))‘^  dt>  + E{V((1  - M(T))‘*’,  R)  - g B(T), 
* 20 


I 


1 


r 

i 


W 


1 

■i 

where  g is  the  maximal  long-run  average  expected  return  per  unit  tine.  i 

Moreover,  the  stationary  policy  associated  with  S(l)  i * | 

U(l-fl)  - U(l)  Is  average- return  optimal  and  attains  the  maximum  In  (20).  | 

Proof.  The  proof  uses  results  of  Llppman  [6],  [8]  and  parallels  that  of 
Theorem  7 of  Llppman  and  Stidham  [9].  First  observe  that,  for  each  a > 0, 

S (1)  > C (1)  “ (1  - (p/ (a+u))^^^)/a  and  that  C (1)  t (l+l)/u,  as  o o+. 

® Cl  ot 

Now  choose  M so  large  that  a(1-F(M})  < p and  choose  1 so  large  that 

(i  + l)/p  > M.  Then  It  Is  possible  to  find  an  a >0  such  that  a < a Implies 

0 '•0 

that  C (1)  > M,  and  hence  C (1)  > C ( 1 ) > M for  all  1 > £.  Co.iiblnlng  these 

QW  sat 

results,  we  conclude  that  there  exists  an  a > 0 and  i < » such  that  ' 

o 

(21)  X (1  - F(S  (1)))  5 X (1  - F(S  (i)))  < p.  for  all  a < a , 1 > i. 

a — o •=  0 “ 

Nov  let  fr^,  n > 1,  be  the  stationary  policy  associated  with  so  chat 

0 

Is  optimal  for  the  Infinite-horizon  problem  with  discount  rate  a “ 

Let  P(v^)  be  the  transition  probability  matrix  for  the  Markov  chain.  Imbedded 
at  arrival  points,  associated  with  n^.  It  follows  from  (21)  that 
(P(n^),  n ^ 0}  Is  uniformly  positive  recurrent,  so  that  starting  In  state  1, 
both  the  expected  time  and  Che  expected  value  of  the  sum  of  customer  rewards 
and  the  absolute  value  of  customer  holding  costs  incurred  until  the  first  visit 
to  state  0 are  uniformly  bounded  in  n.  The  remainder  of  the  proof  Is  Identical 
to  that  of  Theorem  7 of  [9]. 
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OptlMl  Congestion  Tolls  for  the  Infinite  Horizon 


Th«  relAClonshlp  bacman  socially  eptiaal  and  Individually  optlaal 
Joining  pollclas,  discussed  in  Sactlon  2 for  f inlCa-horlzon  probleas, 
carries  over  In  most  respects  to  Inflnlta-horlzon  problene. 

To  see  this,  assume  as  In  Section  2 that  (1),  (11),  and  (111)  hold. 
First  consider  the  problem  with  discounting  (a  > 0)  and  an  Infinite  horizon. 
The  Individually  optimal  policy  is  the  ssme  as  in  the  case  of  a finite 
horizon,  because  of  assumption  (111) : 


enter  Iff  r > C (1) , 
o 

where  C (1)  Is  the  expected  a-discounted  holding  cost  of  a customer  who  enters 
a 

when  1 customers  are  present,  and  Is  given  by  (12).  Define  Tj,(D*  the  optimal 
congestion  toll  for  the  Infinite-horizon  problem  with  discounting,  by 

T^(l)  H S^(l)  - C^(l). 

Since  S (1)  > C (1)  (Theorem  8)  and  S (1)  + S (1)  as  n ® (Theorem  11), 
n,o  “ n,o  o 

we  conclude  that  T (1)  ♦ T (1)  as  n - and  T (i)  > 0,  so  that  once  again 
n,a  a a “ 

a socially  optimal  policy  admits  fcswer  customers  than  an  Individually  optimal 

policy.  Define  U (1)  s U (1)  + h^(i).  Then  U (1)  - lim  U (1)  (Theorem  11) 

and  hence  U^(l)  i 0.  Moreover,  T^(l)  • ~ U^(l+1),  so  that  U^(l)  i 

U (1-H).  Therefore,  U (1)  approaches  a finite  limit  as  l-^»and  hence 
a o 

T (1)  -*•  0 as  1 so  that  the  toll  cannot  be  monotouically  Increasing,  Just 

a 

as  in  the  case  of  a finite  horizon.  Finally,  It  follows  from  (16)  and  the  fact 


that  T (1)  T (i)  as  n -r  • that  T (i)  satisfies  the  functional  equation: 
n,a  a a 


( -«T  S (d-H-N(T)) 
T^(l)-E|e  ^s^((1-n(T))-^) 


*)  ) 
Pr{R>r}dr  j 


E {e““^  T^(1-N(T));  N(T)<  1}  . 
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Hov  consider  what  happens  in  Che  infinite-horizon  aa  a o+  and  the 
criterion  becomes  oaxlmlzatlori  of  long-run  average  expected  return  for  unit 


time.  The  individually  optimal  policy  is: 

i 

i 

enter  iff  r ^ C(i), 

where  C(l)  » 11m  C (1)  - (l+l)/u,  the  expected  holding  cost  of  a customer  who 
a‘^o+  Cl 

enters  when  1 customers  are  (:re8enC.  Define  T(l),  the  optimal  congestion  toll 
for  Che  Infinite-horizon,  average-return  problem,  by 

T(l)  = S(l)  - C(l). 

Since  S (1)  -►  S{1)  (Theorem  12)  and  C (1)  -►  C(l)  as  a -►  o+,  we  conclude  that 
a a 

T(i)  “ lim  T (1)  and  hence  T(l)  ^ 0.  It  follows  from  (22)  that  T(l)  satls- 
<r*o+  a 

fles  the  functional  equation 

H {T(1-N(T));  N(T)  < 1}  , 

If  we  define  g(i)  i E [/s((l-N(T))+r^  Pr{R>r}drJ  and 
b(k)  r Pr{N(T)«U}  , k ^ 0,  then  (23)  can  be  rerwritten  as 

(24)  T(l)  - g(i)  + T(i-k)  b(k), 

k«0 

which  is  a discrete  renewal  equation  (Feller  {3],  p.  290). 

It  follows  from  the  discrete  form  of  the  Key  Renewal  Theorem  (Feller  (3], 
p.  291,  Theorem  1)  that 

(25)  11m  T(l)  - p E{(R-S(0))+}  , 

i-Ko 

where  p"^  - y ET  - E{H(T)}  - Z k b(k),  and 

k-0 

» > E{(R-S(0))+}  - E{/“  Pr{R>r}dr)  - g(l),  by  Fublni's  Theorem. 

i 


I 
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1 


I 
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Now  p > 0 by  assumption  (since  ET  < “)  and  the  case  in  which 
E{ (R-S(O))'^}  ■ 0 is  of  little  practical  interest.  (In  that  case  g(i)  3 0 
and  (24)  implies  that  T(l)  H 0).  Hence  we  conclude  that  in  general  T(l)  has 
a positive  limit  as  i -*’  <»,  in  contrast  to  the  discounted  and/or  finite- 
horizon  cases.  It  is  thus  theoretically  possible  for  T(l)  to  be  monotonically 
increasing.  One  might  hope  to  determine  whether  or  not  1(1)  is  monotonically 
increasing  in  general  by  appealing  to  the  functional  equation  (23),  but  we 
have  been  unable  to  do  so  for  the  case  of  a general  Interarrival-tlme  distri- 
bution. (See  below,  however,  for  an  analysis  of  the  M/H/1  case).  Incidentally, 
while  in  theory  it  should  be  possible  to  use  (23)  or  (24)  (or  a finite  trun- 
cation thereof)  to  solve  explicitly  (or  approximately)  for  the  T(i)'s,  this 
approach  is  complicated  considerably  by  the  fact  that  the  equations  are  not 
linear,  since  the  g(i)'s  depend  on  the  T(i)'s  through  S((i-N(T))'*')  and 
S((1+1-N(T))+).  We  can  answer  the  question  of  monotoniclty  of  the  tolls,  and 
also  give  algorithms  for  their  conqiutatlon,  if  we  specialize  to  the  case  of 
an  exponential  Interarrlval-time  distribution  with  parameter  X - (ET) 
an  M/M/1  system.  In  this  case  the  system  has  the  Markov  property  at  epochs 
of  service-completions  and  "null  events"  (see  Llppman  [7]  and  Lippmsn  and 
Stidham  [9])  as  well  as  arrivals  and  it  can  easily  be  shown  that  the  following 
functional  equations  are  equivalent  to  (20)  for  the  average-return  problem: 


V(i,r) 

U(l) 

U(0) 


max{r+U(l+l),  U(l)} 


xfe  EWl.RH. 


^ 0(1-1)  - ^ . 1 > 1. 


I 
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Moreover,  the  functional  equation  (23)  for  T(l)  Is  equivalent  to 
(27)  T(l)  • p Pr{R>r)dr  + T(l-l) 

(with  T(-l)  0).  We  can  conclude  innedlately  from  (27)  that  T(l)  la  bodo- 

toolcally  Increasing  in  1 In  this  case,  since  S(l-fl)  ^ S(l). 

This  result,  together  with  the  fact  that  the  toll  can  never  be  nonoton- 
Ically  Increasing  when  the  discount  rate,  a.  Is  positive,  leads  us  to  conclude 
that  there  Is  no  strongly  (Blackwell)  optimal  policy  for  this  system.  For,  If 
there  were.  It  would  simultaneously  be  optimal  for  all  sufficiently  small  a > 0 
and  also  optimal  for  the  average-return  problem.  But  this  Is  impossible,  since 
the  characteristics  of  these  two  classes  of  policies  are  mutually  Inconsistent. 
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